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Abstract 

The proposed study is on the partial clustering algorithms for cognitive sensor networks 
that deal with partially observed data. The proposed algorithms aim to estimate clusters in the 
presence of missing values and leverage data imputation techniques to fill in the gaps in the 
target and station device matrices. A modified loss function is introduced to shape the cluster 
centers, and robust Non-negative Matrix Factorization (NMF) algorithms are utilized to 
enhance the robustness of the clustering process. This research contributes to the field of 
cognitive sensor networks by providing insights into the challenges of partial clustering and 
presenting effective algorithms to address them. The proposed methods have the potential to 
enhance the performance of clustering tasks in various domains, including sensor networks, by 


accounting for missing data and producing accurate cluster reconstructions. 
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1. Introduction 


Partial clustering of sensor networks is important for modern communications [1-3]. 
Partial clustering significantly reduces the overhead associated with complete clustering. In 
complete clustering, where all nodes participate in the clustering process, the exchange of 
clustering messages and the reconfiguration of clusters can consume significant network 
resources. With partial clustering, clustering is performed selectively, only when it is 
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necessary, thereby minimizing the overhead and conserving energy. Clustering helps in 
balancing the energy consumption among sensor nodes. By forming clusters, nodes can 
aggregate and merge their data at the cluster head, reducing the amount of data transmitted to 
the base station. Partial clustering further enhances energy efficiency by ensuring that 
clustering is initiated only when the cluster head has consumed a certain portion of its 
energy.This approach helps in prolonging the network's lifetime by efficiently utilizing the 
limited energy resources of sensor nodes. Sensor networks are often deployed on a large scale, 
consisting of hundreds or thousands of nodes. Partial clustering enables scalability by allowing 
the network to handle a large number of nodes. Instead of all nodes participating in the 
clustering process simultaneously, only a subset of nodes needs to be involved in each round 
of clustering. This reduces the computational and communication requirements, making it 
feasible to deploy and manage large-scale sensor networks [4]. Partial clustering allows for 
dynamic network reconfiguration based on changing conditions and requirements. As the 
network evolves over time, the clustering structure can be adjusted to accommodate new nodes, 
node failures, or changes in network topology. By continuously evaluating the energy levels 
and scores of nodes, the partial clustering algorithm can adaptively select new cluster heads 
and members, ensuring efficient utilization of resources and maximizing network performance. 
Partial clustering of sensor networks presents several challenges that need to be addressed. One 
of the key challenges in partial clustering is selecting appropriate cluster heads. The algorithm 
needs to consider factors such as residual energy, node scores, and network topology to identify 
the most suitable nodes to act as cluster heads. Designing an efficient and effective cluster head 
selection mechanism is crucial to ensure optimal cluster formation and energy balancing. 
Partial clustering algorithms operate in a distributed manner, where each sensor node 
independently decides whether to become a cluster head or join an existing cluster. Achieving 
consensus among nodes without global information or centralized control is challenging. The 
algorithm must incorporate mechanisms for nodes to exchange local information, make 
decisions based on that information, and coordinate their actions to form clusters in a 
decentralized manner. Sensor networks are dynamic in nature, with nodes joining or leaving 
the network over time. This poses a challenge for partial clustering algorithms, as the clustering 
structure needs to adapt to changes in node availability and network topology. Ensuring smooth 
transitions during cluster reformation and maintaining network stability in the face of dynamic 


events require careful algorithm design and coordination among nodes. Partial clustering aims 
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to reduce the overhead associated with complete clustering, but finding the right balance 
between overhead reduction and network efficiency is challenging. The algorithm should 
determine the optimal conditions for initiating partial clustering to minimize overhead while 
still maintaining efficient energy utilization and network performance. Striking the right trade- 
off is crucial to achieve energy savings without sacrificing data collection, communication 
reliability, or network lifetime. As sensor networks scale up in size, the challenges of partial 
clustering become more pronounced. The algorithm should be scalable to handle large numbers 
of nodes and should not be overly complex in terms of computational requirements or 
communication overhead. Ensuring scalability while maintaining the benefits of partial 
clustering, such as energy efficiency and adaptability, is a significant challenge in the design 
and implementation of clustering algorithms. Addressing these challenges requires a thorough 
understanding of the network characteristics, careful algorithm design, and performance 
evaluation. Researchers and engineers need to consider these challenges to develop robust and 
efficient partial clustering algorithms that can effectively support the requirements of diverse 
sensor network applications. To address the challenges of partial clustering in sensor networks, 
several solutions can be considered. Develop sophisticated algorithms for cluster head selection 
that take into account multiple factors such as residual energy, node scores, communication 
costs, and network topology. Machine learning techniques, optimization algorithms, or hybrid 
approaches can be employed to improve the accuracy and efficiency of cluster head selection. 
These advanced methods can provide better cluster formation and energy balancing. Design 
algorithms that adapt to changes in network dynamics, such as node mobility, failures, or 
additions. Introduce mechanisms for periodic re-evaluation of cluster formation to 
accommodate new nodes or changes in node characteristics. Incorporate dynamic decision- 
making processes that consider real-time information to adjust cluster configurations, ensuring 
the algorithm remains effective and efficient in dynamic environments. Develop efficient 
communication protocols to minimize the overhead associated with clustering messages. 
Utilize techniques such as data aggregation, compression, or hierarchical communication 
structures to reduce the amount of data transmission and clustering-related signalling. Efficient 
communication protocols can reduce energy consumption and improve network scalability. 
Combine partial clustering with other techniques such as data-driven approaches, machine 
learning, or mobility-aware algorithms [5]. Hybrid approaches can enhance the clustering 


process by leveraging additional information or context-specific factors. For example, 
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incorporating data similarity metrics or location-based information can improve the accuracy 
of cluster formation and enable better utilization of sensor resources. Implement energy-aware 
scheduling mechanisms that optimize the active and sleep periods of sensor nodes. Design 
protocols that allow nodes to coordinate their active periods to ensure data collection and 
communication while maximizing energy conservation. By synchronizing the active periods, 
nodes can effectively utilize available resources and reduce energy wastage. Conduct extensive 
simulations and performance evaluations to assess the efficiency and effectiveness of partial 
clustering algorithms. Consider various network scenarios, such as different network sizes, 
node densities, and traffic patterns, to evaluate the scalability, energy efficiency, network 
lifetime, and data reliability of the algorithms. This iterative process helps refine and optimize 
the algorithm design. Foster collaboration among researchers, practitioners, and 
standardization bodies to establish common guidelines and benchmarks for partial clustering 
algorithms. Encouraging the exchange of ideas, sharing of datasets, and promoting open 
research can accelerate advancements in partial clustering techniques and facilitate the 
adoption of standardized solutions. By exploring these possible solutions and continuously 
advancing research and development in partial clustering, it is possible to overcome the 
challenges and achieve more efficient and scalable sensor networks with prolonged network 


lifetime and improved energy utilization. 


In summary, the scope of the proposed work is to estimate clusters in the presence of 
missing values and leverage data imputation techniques to fill in the gaps in the target and 


station device matrices. Therefore, clustering models can be performed. 


The rest of this study is organized as follows. Section 2 shows related works. Section 3 


is the proposed method with the experiment in Section 4. Section 5 is the conclusion. 


2. Related Work 


Non-Negative Matrix Factorization (NMF) is a matrix factorization technique that has 
been successfully applied in various domains, including signal processing, image processing, 
and clustering [6-16]. While NMF may not be directly applicable to the specific problem of 
partial clustering in sensor networks, it can be leveraged as a component or an optimization 
technique within a larger clustering framework [17, 18]. NMF can be employed to extract 


meaningful features from sensor data. By decomposing the sensor data matrix into non- 
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negative basis vectors, NMF can identify underlying patterns or components in the data. These 
extracted features can then be used as input for subsequent clustering algorithms. NMF can be 
used for data compression by approximating the original data matrix with a lower-rank NMF 
representation. This can help reduce the amount of data that needs to be transmitted or 
processed, leading to energy savings and improved efficiency in partial clustering algorithms. 
NME can facilitate dimensionality reduction by identifying a reduced set of non-negative basis 
vectors that capture the most significant characteristics of the data. By reducing the 
dimensionality of the data, clustering algorithms can operate on a smaller feature space, which 
can enhance clustering performance and reduce computational complexity. NMF can be 
utilized to initialize the cluster centers or prototypes in clustering algorithms. By decomposing 
the data matrix into non-negative components, NMF can provide an initial set of representative 
patterns or cluster centers. These initial centers can then be refined and updated using 
traditional clustering algorithms to achieve better clustering results. NMF-based clustering 
algorithms can be developed by incorporating NMF directly into the clustering process. These 
algorithms aim to simultaneously cluster the data while decomposing it into non-negative 
components. By integrating NMF and clustering, it is possible to exploit the inherent non- 
negativity constraints of the data and potentially improve clustering accuracy and 


interpretability. 


L2-norm NMF (Nonnegative Matrix Factorization) is a variant of the traditional NMF 
algorithm that uses the L2-norm (Euclidean norm) as the regularization term in the objective 
function [19, 20]. The objective of L2-norm NMF is to find nonnegative factor matrices that 
minimize the reconstruction error while promoting sparsity and stability in the factorization. In 
the standard NMF formulation, the objective function is typically based on the Frobenius norm, 
which is the L2-norm of the difference between the original data matrix and its approximation 
reconstructed from the factor matrices. However, L2-norm NMF extends this by adding an L2- 
norm regularization term to the objective function. The Le-norm regularization term 
encourages sparsity in the factor matrices by penalizing large values and promoting small 
values. This helps in identifying a concise and meaningful set of components that contribute 
significantly to the data representation while suppressing the influence of less important or 
noisy components. By incorporating the L2-norm regularization, Lo-norm NMEF can help 


improve the robustness, generalization, and interpretability of the factorization results. 
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Li-norm NMEF (Nonnegative Matrix Factorization) is a variant of NMEF that 
incorporates the Li-norm (also known as the Manhattan norm or absolute norm) as the 
regularization term in the objective function. Li-norm NMF promotes sparsity in the factor 
matrices, resulting in a more sparse and interpretable representation of the data. In the standard 
NMEF formulation, the objective function is typically based on the Frobenius norm, which is 
the Lo-norm of the difference between the original data matrix and its approximation 
reconstructed from the factor matrices. However, Li-norm NMF extends this by adding an L1- 
norm regularization term to the objective function. The Li-norm regularization term 
encourages sparsity by promoting coefficients or entries of the factor matrices to be exactly 
zero. This leads to a sparse representation, where only a subset of components is actively 
involved in explaining the data. By incorporating the Li-norm regularization, Li-norm NMF 
encourages sparse and concise representations, where only a few components are selected to 
explain the data, while the majority of components have zero or near-zero coefficients. This 
Sparsity property can enhance interpretability, reduce overfitting, and improve the robustness 
of the factorization results. Li-norm NMF has been widely used in various applications such 
as signal processing, image analysis, text mining, and feature selection, where sparsity and 


interpretability are desired. 


L21-norm NMF (Nonnegative Matrix Factorization) is an extension of NMF that 
utilizes the L2,1-norm as a regularization term in the objective function [21, 22]. The L2,1-norm 
promotes structured sparsity, which encourages groups of coefficients or entries in the factor 
matrices to be simultaneously zero or small. In the standard NMF formulation, the objective 
function is typically based on the Frobenius norm, which is the L2-norm of the difference 
between the original data matrix and its approximation reconstructed from the factor matrices. 
However, L2,1-norm NMF extends this by adding an L2,1-norm regularization term to the 
objective function. The Lz21-norm regularization term promotes structured sparsity by 
encouraging groups of coefficients or entries in the factor matrices to be jointly zero or small. 
This means that instead of having individual coefficients set to zero independently, entire 
groups or subsets of coefficients are simultaneously set to zero. This can lead to more 
interpretable and meaningful structure in the learned representations. The L2,1-norm is 
computed as the sum of the L2-norms of the rows of a matrix. In the context of NMF, the L2,1- 
norm of the right factor matrix encourages sparsity across the columns of the right factor 


matrix, promoting group sparsity in the factorization. By incorporating the L2,1-norm 
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regularization, L21-norm NMEF encourages structured sparsity, where entire groups of 
coefficients are simultaneously set to zero or small. This can be useful in scenarios where there 
is prior knowledge or assumption about the underlying structure of the data, and can lead to 
more meaningful and interpretable factorizations. L2,1-norm NME has been applied in various 
domains such as image processing, bioinformatics, and text mining, where structured sparsity 


and group-level interpretations are desired. 


Robust NMF (Nonnegative Matrix Factorization) is an extension or variant of the 
traditional NMF algorithm that incorporates robustness mechanisms to handle outliers, noise, 
or corrupted data [23-31]. It aims to improve the stability and reliability of the factorization 
process in the presence of disturbances in the input data. The basic NMF algorithm assumes 
that the input data can be accurately represented as a linear combination of nonnegative 
components. However, in real-world scenarios, the data may contain outliers or noise that can 
significantly impact the factorization results. Robust NMF algorithms address this issue by 
introducing robust optimization techniques or incorporating additional constraints to enhance 
the resilience of the factorization process. There are different approaches to achieving 
robustness in NMF. Some common techniques include: Robust cost functions: Instead of using 
the traditional least squares error as the objective function, robust NMF algorithms employ 
robust cost functions that are less sensitive to outliers. Examples include L; norm, Huber loss, 
or Cauchy loss functions[32-35]. By imposing sparsity constraints on the factor matrices, 
robust NMEF algorithms encourage the identification of a sparse set of meaningful components 
while suppressing the impact of outliers or noise. Robust NMF methods may include outlier 
detection and rejection mechanisms to identify and discard outliers or noisy samples before 
performing the factorization. This helps improve the accuracy of the factorization results. The 
goal of robust NMEF is to achieve more reliable and interpretable factorizations, even in the 
presence of challenging data conditions. By considering the robustness aspect, these algorithms 
can be valuable in various domains, such as image processing, signal analysis, bioinformatics, 


and data mining, where data quality and integrity are critical factors. 
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3. Proposed Work 


Proposed System 


Regularization 


| 


Target Device Matrix . : 

Nonnegative Matrix : Cluster 
Ce Factor Formation ; 

Cofactorization Reconstruction 


Station Device Matrix 


Figure 1. Framework of the Proposed System 


The overall framework of the proposed system is illustrated in Figure 1. The inputs are target 
and station device matrices, which are analysed in the co-factorization module while 
regularization constraints are imposed. Then, the factor matrices are computed and used for 


cluster reconstruction. The details are as follows. 


Let M be the number of feature dimensions, L mean the number of targets, and N signify the 
number of station devices. Target and station device matrices Xr and Xp are M-by-L and M- 


by-N, respectively. Assume that they contain missing value entries. Data imputation algorithms 
are used to fill in the entries with substituted values and obtain estimated X, and X, . Then, 


the loss function is defined as (1) based on [15] with some modifications to fit the applications. 
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where W, Hp, and Hr are factor matrices, and their dimensions are M-by-P, P-by-N, and P- 
by-L. Additionally, ||-||r, ||-[|2, |[-||1, and |[-||2,1 are the Frobenius norm, L2 norm, Li norm, and 
L2,1 norm, Tr(-) is the trace operator, T is the transpose, and * is the slicing operator. Moreover, 
a, f, y, 6, and J are all scalar parameters to control penalty terms. Finally, A and B are a 
similarity matrix and a connection bipartite matrix, and P is the hidden dimension. Cluster 
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centers are formed in W. To increase robustness, robust NMF algorithms are used to reshape 


cluster centers by using half quadratic loss functions ¢(.). Then, 


An 


min{4(X, ~UV,)+4(X;—-UV,) +7, |UJ, +7 [Ul]; + Vol(U)}, (2) 


where 7 is a scalar parameter, and Vol(-) is the volume penalty term. Cluster centers are formed 


in M-by-P U. To fuse cluster center information, the weighted function is used. 
C=W%(W+U)+U%(W+U), (3) 


where % is elementwise division. The algorithm 1 below illustrates the robust NMF algorithms 


used in reshaping the cluster centers by using half quadratic loss functions. 


Input: Target and station device matrices Xr and Xp 

Output: C 

Initialize W, Hp, and Hr 

Compute A and B 

Do While 
Compute W using the multiplicative update rule 
Compute Hp using the multiplicative update rule 
Compute Hr using the multiplicative update rule 

Until Convergence 

Reconstruct X, using WHp 


ONDNFPWNR 


\o 


Reconstruct X, using WHr 
Do While 
Compute U using NNLS 
Compute Vp using NNLS 
Compute Vr using NNLS 
Until Convergence 
Compute C using W and U 


Tet cell eel ee ll ee ce 
nABRWNR CO 


Algorithm .1 Cluster Centre Reshaping 


The intuition behind the proposed method was that the factor matrices and coefficient matrices 
learned from the inputs (although they were incomplete) are used. The defect parts could be 
initialized and updated in iterations (like the Expectation-Minimization strategy). Therefore, 


one could estimate the correct content inside the incomplete part. 
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Data imputation plays a crucial role. Data imputation techniques are used to address the 
issue of missing values in the collected data. Missing data can arise due to various reasons, 
such as sensor failures, transmission errors, or limitations in data collection mechanisms. The 
absence of data points can significantly impact the quality of cluster analysis, making data 
imputation a necessary step to mitigate this problem. Necessity of data imputation is as follows: 
it's common to have missing data because not all sensors or devices report information 
simultaneously. Imputing missing values ensures that data integrity is maintained and that 
clusters are formed based on as much available information as possible. Data imputation 
enhancing cluster quality. Accurate and complete data are essential for producing meaningful 
clusters. Imputing missing data allows the clustering algorithms to consider all relevant 


information, leading to more accurate cluster assignments. 


4. Results and Discussion 


To evaluate the effectiveness of the proposed method, this study conducted experiments 
using real sensor networks to collect sensing data. The experiment involved a total of 10 targets 
and 20 stations, implemented on embedded systems comprising transmitter and receiver 
components. Specifically, this study used embedded systems “Raspberry Pi 4B + Waveshare 
RMS502Q-AE 5G HAT” to collect data and MATLAB to simulate clustering reconstruction 
environments. Each target device transmitted labelled data to the corresponding station, with a 
total of two unique labels. It is important to note that each target device was associated with 
only one label type, ensuring clear distinctions. At lease one connection between a target and 
a station was established. To simulate real-world scenarios, this study introduced missing 
values into the collected data matrices. The missing values were randomly selected, resulting 
in a missing rate of 5.00%. Additionally, all labels were removed from the data matrices. To 
address the missing values, this study employed a data imputation algorithm, generating 
imputed matrices for further analysis. In the evaluation, this study varied the variable p, which 
represents a parameter range from two to 10, with a fixed separation of one. It is worth 
mentioning that the correct number of clusters in the data was known to be two. After the 
proposed method estimated the number of clusters, this study used accuracy rates to assess its 


effectiveness in producing accurate results. 
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The procedure for selecting favourable values of the model parameter was based on 
layers of loops, where one loop was designed for one parameter. Five parameters involved five 
layers. This study recorded the performance of the model with respect to each change in 
parameter. The step size was 0.50, and the range was within one and zero. When the loops 


ended, the system could find the best ones. 


Table 1. Favourable Values of the Parameters 


Parameter Favourable Value 
a 0.30 
B 0.50 
y 0.25 
O 0.30 
A 0.50 


Experiment 1: Assessment of the Need for Data Imputation 


This study performed a test to assess the necessity of data imputation (with the use of 
Robust NMEF [32], ie., filling in missing data with initial values and iteratively updating 
replacements after matrix factorization and rebuilding as shown in Figure 2.using four distinct 


datasets collected with the devices. 


Incomplete Fill in Missing Entries dstits Pacentoniiog Matrix Rebuilding 
Data Matrix with Initial Values Factor | by the Product of Factors 


Matrix 


a 
Replacing the Matrix < Converged — 
Values eo a 


Ty Yes 


Figure 2. Data Imputation Flow 


The procedure followed the same protocol mentioned at the beginning of this section. The baseline was 
the proposed method without data imputation. The experimental results are presented in Figure 4. The 
vertical axis represents clustering accuracy, while the horizontal axis displays different datasets. It is 
evident that the proposed algorithm with data imputation produced superior results. The figure.3 


illustrates the accuracy comparison of the need for data imputation. 
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Figure 3. Accuracy Comparison of the Need for Data Imputation 


Experiment 2: Assessment of Different Methods 


This study conducted a performance evaluation test comparing the proposed method 
with several existing methods, using the same datasets and parameters as before. The methods 
for comparison included Comodule Discovery (CD) [15], Cooperative Comodule Discovery 
(CCD) [5], Versatile Clustering (VC) [14], and Alternate Least Squares (ALS) [17, 18]. Figure 
4 displays the numerical results, with the horizontal axis representing different datasets and the 
vertical axis representing accuracy. As shown in the figure, this study observed that the 
proposed method improved accuracy by an average of 5.99% across all the datasets, 
demonstrating its effectiveness. The improvements were 7.24%, 4.58%, 6.05, and 6.11% for 


each dataset. 
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Figure 4. Accuracy Comparison of Different Methods. 
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Experiment 3: Testing of Cluster Prediction 


The experimental results are depicted in Figure 5, with accuracy represented on the 
vertical axis and the number of clusters on the horizontal axis. It is evident from the graph that 
the highest accuracy aligns with the correct number of clusters. This clear correlation indicates 


the effectiveness of the proposed method in accurately reconstructing clusters. 
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Figure 5. Experimental Results of Estimated the Number of Clusters 


5. Conclusion 


This study proposes a method for estimating clusters in cognitive sensor networks with 
partially observed data. The method involves utilizing target and station device matrices with 
missing value entries, which are filled using data imputation algorithms. A loss function is 
defined based on factor matrices and penalty terms to shape the cluster centers. Robust NMF 
algorithms are employed to increase robustness. To evaluate the proposed method, experiments 
were conducted using real sensor networks. The experiment involved 10 targets and 20 stations, 
and missing values were introduced into the data matrices with a 5.00% missing rate. Labels 
were removed from the matrices, and a data imputation algorithm was used to generate imputed 


matrices. The evaluation included varying a parameter range from two to 10, with the correct 
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number of clusters known to be two. The accuracy rates were used to assess the effectiveness 
of the method. The experimental results, illustrated in Figure 5, demonstrate that the highest 
accuracy corresponds to the correct number of clusters. This indicates the effectiveness of the 


proposed method in accurately reconstructing clusters. 
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